Anomaly Intrusion Detection by Internet Datamining of Traffic Episodes*

We present a new datamining approach to generating frequent episode rules for building anomaly-based, intrusion detection systems. The episode rules are generated to detect anomalous sequences of TCP, UDP, or ICMP connections, which deviate from normal traffic episodes. Rule pruning techniques are introduced to reduce the search space by 40-70%. The new method demonstrates its effectiveness in detecting unknown network attacks embedded in traffic connections for common Internet services like telnet, http, ftp, smtp, Email, authentication, etc. The new approach accelerates the entire detection process of machine learning and profile matching. Testing our scheme over 10 days of real-life Internet datasets traced at USC and mixed up with the MIT/LL attack data, we encountered less than 20 false alarms over 200 network attack incidents. Our frequent-episode scheme results in an intrusion detection rate up to 47% for DoS (denial of service), 19% for R2L (remote-to-local), and 40% for port-scanning Probe attacks. These results demonstrate an average of 51% improvement over the use of association rules alone. Our scheme detects many novel attacks that cannot be detected by Snort, including the Smurf, Apache2, Guesstelnet, Dict, Neptune, Udpstorm, etc. We recommend the use of anomaly detection scheme jointly with signature-based IDS to build future intrusion defense system in real time.